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Preface 


The  GRLL  was  developed  as  a  flexible,  realistic  research  environment  that 
would  permit  the  investigation  of  the  basic  utility  of  Group  Support  Systems  within  the 
military  acquisition  environment.  In  this  paper,  we  present  the  basic  design 
considerations  for  the  GRLL,  data  on  its  initial  use,  and  lessons  learned  in  the  areas  of 
technology  design  and  employment  within  the  Air  Force  environment. 

This  work  supports  the  Armstrong  Laboratory,  Logistics  Research  Division, 
Acquisition  Logistics  Branch's  (AL/HRGA)  ongoing  work  in  the  area  of  developing  and 
demonstrating  various  integrated  tools  and  techniques  to  aid  in  implementing 
Integrated  Product  Development  (IPD)  and  support  the  in-house  capability  to  perform 
research  and  development  in  design  decision  support,  information  technology  and 
information  integration  for  weapon  system  requirements  development  and  product 
design  (work  unit  number  1710-00-18). 
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Introduction 

Until  recently,  computers  have  been  used  primarily  to  support  individuals 
working  on  individual  tasks.  The  availability  of  new  technologies  has  enabled 
researchers  and  system  developers  to  explore  the  use  of  networked  computers  to 
support  teams  doing  group  work.  Group  support  systems  (GSS)  have  been  developed 
to  help  teams  with  a  variety  of  tasks,  ranging  from  sharing  ideas,  organizing 
alternatives,  negotiating  for  consensus,  and  voting  on  options,  leading  toward  the 
realization  of  team  goals.  GSS  have  been  shown  to  increase  the  effectiveness  and 
efficiency  of  teams  by  supporting  and  enhancing  beneficial  group  processes,  while 
circumventing  or  minimizing  many  of  the  counter-productive  aspects  of  group  work 
[Heminger  1989,  Nunamakeret  al.  1989]. 

Supporting  teamwork  is  of  significant  interest  to  many  organizations,  including 
the  Air  Force,  because  of  the  central  role  teams  perform  in  today's  organizational  life. 
To  explore  this  area,  the  Air  Force  Armstrong  Laboratory,  Logistics  Research  Division, 
(AL/HRG),  at  Wright-Patterson  Air  Force  Base  sponsored  the  development  of  the 
Group  Research  Laboratory  for  Logistics  (GRLL).  We  created  this  facility  to  conduct 
research  into  the  applicability  of  GSS  technology  to  Air  Force  (AF)  teams,  to  better 
understand  both  the  design  of  multi-functional  teams  and  team-enhancing 
technologies,  and  to  understand  more  fully  the  role  of  human  facilitation  and  the 
feasibility  of  employing  nonhuman  facilitation  agents  in  a  GSS  environment.  In  this 
paper,  we  present  the  basic  design  considerations  for  the  GRLL,  data  on  its  initial  use, 
and  lessons  learned  in  the  areas  of  technology  design  and  employment. 

Organizational  problem  solving  makes  extensive  use  of  groups  and  teams. 
(Although  some  researchers  distinguish  between  these  terms,  differences  are  not 
important  within  the  context  of  our  current  GSS  research.)  As  with  most  large 
organizations  today,  there  are  a  number  of  reasons  for  the  extensive  use  of  teams 
within  the  Air  Force;  organizational  problems  can  be  very  complex,  and  they  may 
require  input  of  from  a  variety  of  experts  and  group  "buy-in"  or  endorsement  of  selected 
solutions.  The  consideration  of  these  reasons  leads  to  the  exploration  of  teamwork  for 
problem  solving. 

Most  organizations,  including  the  AF,  recognize  the  value  of  teamwork.  For 
example.  Air  Force  Chief  of  Staff,  General  Merrill  A.  McPeak  has  said  that,  "Teams  are 
the  key  to  quality  and  innovation"  [1993].  It  is  widely  accepted  that  extending 
computers  to  support  teamwork  holds  great  promise.  Even  a  partial  realization  of 
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promised  effectiveness  and  efficiency  gains  would  have  great  value  for  organizations 
that  already  value  teamwork. 

Recent  technological  advances  (e.g.,  those  permitting  greater  connectivity  and 
those  improving  the  human-computer  interface),  coupled  with  organizational  needs, 
have  stimulated  researchers  and  system  developers  to  explore  the  use  of  networked 
computers  to  support  the  work  of  teams. 

The  central  role  of  teamwork  within  the  Air  Force  compels  it  to  seek  methods 
that  will  improve  team  productivity.  Research  that  led  to  the  creation  of  the  GRLL  was 
undertaken  at  Armstrong  Laboratory  located  at  Wright-Patterson  Air  Force  Base 
(WPAFB).  This  base  is  also  the  headquarters  for  the  Air  Force  Materiel  Command 
(AFMC),  one  of  ten  major  commands  within  the  AF,  with  logistics  responsibilities 
throughout  the  AF.  AFMC  researches,  develops,  provides,  and  supports  all  AF 
weapons  systems,  from  cradle  to  grave,  controlling  a  multi-billion  dollar  annual 
acquisition  budget. 

The  GRLL  was  developed  as  a  flexible,  realistic  research  environment  that 
would  permit  the  investigation  of  the  basic  utility  of  GSS  within  the  military  acquisition 
environment.  In  this  paper,  we  present  the  basic  design  considerations  for  the  GRLL, 
data  on  its  initial  use,  and  lessons  learned  in  the  areas  of  technology  design  and 
employment  within  the  Air  Force  environment. 

Background 

While  computers  are  ubiquitous  in  today’s  offices,  they  are  typically  used  to 
support  individual  workers  performing  individual  tasks  [Stefik  et  al.  1987].  Yet,  studies 
indicate  that  office  workers  spend  from  30  to  70  percent  of  their  time  in  meetings 
[Panko  1964].  With  this  much  of  the  office  workers’  time  spent  in  coilaborative  work 
efforts,  it  would  be  valuable  to  be  able  to  bring  the  power  of  computers  to  bear  on  the 
needs  of  the  work  group.  The  effort  to  meet  this  need  has  led  to  research  in  the  area  of 
group  support  systems  (GSS).  A  GSS  is  defined  as  a  computer-based  system, 
comprised  of  a  facility,  hardware,  software,  procedures,  and  facilitation,  which  is 
designed  to  support  and  augment  the  efforts  of  work  groups  to  complete  unstructured"' 
tasks  [Heminger  1989]. 

GSS  research  has  received  much  attention  in  recent  years,  with  many  articles 
appearing  in  the  information  systems-related  literature.  Programs  of  research  have 


'Unstructured  tasks  are  those  tasks  which  are  non-algorithmic  in  nature.  That  is,  the  tasks  do  not  have  a 
rigorous  solution  procedure  that  will  produce  a  specific,  correct  answer. 
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been  undertaken  at  the  University  of  Arizona  [Dennis  et  al.  1993,  Jessup  et  al.  1988, 
Nunamaker  et  al.  1987;  Nunamaker  et  al.  1988;  Nunamaker  et  al.  1989],  the  University 
of  Minnesota  [DeSanctis  1988;  DeSanctis  and  Dickson  1987;  DeSanctis  and  Gallupe 
1985;  DeSanctis  and  Gallupe  1987;  Gallupe  1985;  Gallupe  1988  et  al.;  Gallupe  et  al. 
1987],  and  at  other  schools  [Bui  1987;  Bui  and  Jarke  1986;  Hiltz  and  Turoff  1985; 
Huber  1981 ;  Huber  1984].  Some  of  these  programs  have  a  distinct  technology 
emphasis  while  others  seek  to  understand  process  issues. 

However,  as  noted  by  Kraemer  and  King  [1988],  the  field  is  still  not  well 
developed.  Most  of  the  empirical  reporting  has  focused  on  GSS  applied  to  small 
groups  in  either  experimental  or  developmental  laboratory  conditions.  Extrapolating 
the  results  of  experimental  group  research  to  real  world  settings  is  inherently 
problematic  because  of  the  large  number  of  uncontrollable,  interacting  variables  in  real 
group  settings.  Kraemer  and  King  suggest  that  examination  of  implementations  of 
group  decision  support  systems  in  operational  environments  would  provide  valuable 
knowledge  about  "the  promises,  problems,  and  challenges  involved  in  this  technology." 

To  date,  the  focus  of  GSS  has  been  on  the  needs  of  business  teams  such  as 
strategic  planners,  product  concept  developers,  and  business-problem  solving  teams. 
Demonstrated  benefits  of  GSS  in  these  settings  include  shorter  meeting  times,  more 
productive  meetings,  increased  participant  satisfaction,  reduced  project  completion 
time,  larger  solution  sets  from  which  to  develop  possible  solutions,  opportunities  for 
innovative  problem  solutions,  increased  buy-in  to  the  problem  solution  by  participants, 
and  increased  awareness  of  decision  making  rationale  [Heminger  1989,  Dennis  et.  al. 
1990].  However,  there  are  many  questions  about  the  use  of  GSS  in  the  military 
environment.  Will  the  same  benefits  that  have  been  seen  in  the  business  world 
translate  to  the  military  environment?  Do  differences  in  the  military  world  require 
modification  of  current  systems,  or  will  these  differences  mandate  new  architectures? 
Differences  in  organizational  structure  between  civilian  and  military  organizations  may 
impact  the  efficacy  of  GSS.  The  more  structured  military  chain  of  command  may  affect 
the  way  that  GSS  are  used  and  may  affect  the  outcomes  produced  through  GSS- 
supported  activities.  Facilitation  and  forced  geographic  dispersion  of  participants  may 
influence  the  best  course  of  adoption  of  GSS  technology.  The  GRLL  was  created  to 
investigate  the  use  of  GSS  in  a  specifically  military  context  that  includes  operational 
procedures  and  outcomes,  development  of  GSS  standards,  and  technology  diffusion 
within  the  military  environment. 


Decision  Room  Faciiities 

A  commonly  used  GSS  is  usually  designed  to  operate  in  a  decision  room 
environment.  Typically,  it  includes  10  to  24  networked  computers  that  run  software 
designed  to  assist  a  group  in  undertaking  basic  problem  solving  processes:  gathering 
ideas,  organizing  ideas,  negotiating,  and  then  making  a  selection  such  as  with  a  rank- 
order  voting  technique. 

One  of  the  earliest  computer-supported  decision  rooms  was  created  in  the  late 
1970s  by  Gerald  Wagner  at  Execucom  [Gibson  and  Lud1 1988].  This  system 
incorporated  a  table  with  eight  participant  terminals,  a  lab  moderator  terminal,  and  a 
large  front  screen,  all  connected  to  a  Prime  400  computer.  The  software  for  the 
system,  called  Mindsight™,  was  designed  to  facilitate  communication,  elicit  ideas,  share 
information,  analyze  data  quantitatively,  and  measure  group  consensus.  The  system 
could  accommodate  both  attributed  and  anonymous  comments. 

Although  the  decision  room  supported  by  Mindsight™  had  most  of  the  features  of 
today's  GSS,  it  was  not  a  successful  effort.  The  project  began  with  high  expectations, 
but  over  a  period  of  three  years,  the  room  fell  into  disuse  and  was  eventually 
dismantled.  Gibson  and  LudI  concluded  that  this  effort  failed,  not  because  of  the 
technical  shortcomings  of  the  system,  but  largely  because  of  management's  insistence 
that  all  ideas  be  attributed  (i.e.,  with  the  submitter’s  name  attached).  They  found  that 
users  saw  attribution  as  unfair  coercion  and  that  users  resisted  using  this  version  of  a 
GSS.  These  findings  were  an  early  indication  of  the  importance  of  specific  features  of 
a  GSS  such  as  anonymity  and  management  influence  over  meeting  process. 

The  University  of  Arizona's  first  GSS  facility  was  created  in  1 986.  This  ten- 
station  facility,  with  networked  computers  at  each  station,  had  a  large  public  screen  and 
linked  all  components  by  a  local  area  network  (LAN).  A  human  facilitator  controlled  the 
software  and  ran  the  meeting.  Although  the  impetus  for  the  creation  of  this  room  was  to 
support  the  development  of  requirements  definitions  for  systems  analysts,  it  was 
quickly  seen  to  have  much  wider  application.  Groups  of  many  kinds,  performing  many 
different  tasks,  could  also  benefit  from  using  this  system. 

The  creators  of  this  system  perceived  the  system's  commercial  viability  and  a 
commercial  spin-off  from  the  university,  Ventana  Corporation,  was  created  to  further 
develop  and  market  what  became  known  as  GroupSystems.  This  popular  system  has 
undergone  several  upgrades  and  is  currently  available  as  a  DOS-based  product, 
GroupSystems  V™.  A  Microsoft  Windows™-based  version  has  been  developed  and  is 

just  now  being  shipped. 
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IBM  supported  the  early  development  of  GroupSystems.  At  one  time,  IBM 
marketed  its  own  version  of  Ventana's  software  under  the  tradename  TeamFocus™. 
IBM's  first  GSS  facility  was  created  in  1987  at  its  plant  in  Owego,  New  York. 

Experience  with  this  facility  was  satisfactory,  and  over  the  next  few  years,  several 
dozen  similar  facilities  were  created  at  other  IBM  plants  around  the  country,  using  first, 
GroupSystems,  and  later,  TeamFocus. 

At  this  time,  many  operational  decision  rooms  are  in  existence  around  the 
country  and  are  using  systems  like  GroupSystems  V.  Most  of  them  are  in  private 
commercial  settings,  but  a  few  are  installed  at  military  installations,  including  the 
Washington  Navy  Yard  in  Washington  D.C.  and  Redstone  Arsenal  in  Huntsville, 
Alabama.  However,  AF  use  of  this  technology  has  been  minimal.  A  decision  room 
facility  was  developed  at  Wright  Patterson  Air  Force  Base  (WPAFB)  to  explore  the  use 
of  this  technology  in  an  AF  setting. 

Development  of  the  Group  Research  Laboratory  for  Logistics 

Demonstrated  effectiveness  in  a  number  of  commercial  settings  and  the  Air 
Force's  interest  in  improving  work  undertaken  by  teams  prompted  our  decision  to 
investigate  the  use  of  GSS  within  the  Air  Force  environment.  The  Logistics  Research 
Division  (AL/HRG)  also  recognized  emergent  concurrent  engineering  process  as  a  shift 
to  group-oriented  rather  than  individually  focused  design  processes.  Analysis  of 
empirical  research  continued  to  suggest  that  a  business  process  focus  was  both  a 
feasible  and  potentially  profitable  research  domain. 

Initially,  in  the  summer  of  1991 ,  AL/HRG  had  little  more  than  a  GSS 
demonstration  capability.  It  consisted  of  six  laptop  computers,  equipped  with  Ethernet 
cards  and  connected  by  a  Novell  NetWare®  LAN.  One  of  the  laptops  with  a  120 
megabyte  hard-drive  served  as  the  network  server.  The  first  two  GSS  software 
systems  tested  were  a  demonstration  version  of  VisionQuest™  and  a  prototype  system 
from  Indiana  University. 

Results  from  the  demonstration  facility  were  positive,  so  we  decided  to  explore 
the  creation  of  a  larger  facility  (i.e.,  the  GRLL)  that  could  support  larger  meetings.  This 
would  open  up  our  research  to  a  much  larger  share  of  the  meetings  that  take  place  in 
the  WPAFB  community.  A  group  facilitator  from  the  Total  Quality  office  estimates  that 
more  than  two-thirds  of  all  groups  they  worked  with  had  six  or  more  members,  and  a 
GSS  that  could  support  ten  participants  could  support  about  90  percent  of  all  their 
groups.  Ten  work  stations  also  matches  well  with  many  of  the  other  GSS  facilities  that 
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have  been  created  in  other  locations  [Wagner  et  al.  1993],  so  we  decided  to  create  a 
ten-station  facility. 

We  selected  GroupSystems  V  from  Ventana  Corporation  of  Tucson,  Arizona,  as 
the  GSS  software.  This  commercial  grade  software  supports  major  functions  of  goal- 
oriented  group  work  (e.g.,  idea  generation,  idea  organization,  and  decision  making). 
Novell  NetWare  was  used  as  the  LAN  software. 

For  hardware,  we  selected  notebook  size  computers  because  of  the  potential 
value  of  portability.  However,  reasonably  priced  notebook  computer  displays  at  this 
time  were  monochrome  only  and  were  of  unsatisfactory  contrast  for  regular  use. 
Concern  was  also  raised  about  the  usability  of  the  small  notebook  keyboards.  In  the 
interest  of  usability,  we  have  included  external  keyboards  and  color  monitors. 

To  complete  the  facility,  we  purchased  workstation  desks  and  chairs,  adjustable 
lighting,  and  improved  air  conditioning.  Workstation  computer  tables  with  a  28-inch 
height  (versus  30  inches  for  a  standard  desk)  were  selected  to  provide  optimal  access 

to  the  keyboard  and  to  allow 
more  room  for  participants  to 
see  each  other  over  the  tops  of 
the  monitors.  For  chairs,  we 
selected  adjustable,  cloth 
upholstered,  swivel  armchairs. 

The  GRLL,  located  in 
Building  434  of  WPAFB 
became  fully  operational  in 
February  of  1993.  Like  many 
GSS  facilities,  the  GRLL  is  a 
dedicated  facility  with  zoned  air 
conditioning  and  lighting.  It 
contains  desks,  networked 
computers,  a  large  public 
screen,  white  boards,  and 
other  features  that  support 
team  meetings  (see  Figure  1). 
Raised  flooring  provides 
channels  for  under-floor  wiring. 
The  normal  GRLL 

Figure  1 .  Group  Research  Laboratory  for  Logistics  occupies 
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approximately  720  square  feet  of  space  (30  feet  deep  by  24  feet  wide)  within  a  1300- 
square-foot  room.  A  movable  back  wall  allows  the  GRLL  to  be  expanded  for  larger 
groups.  In  normal  configuration,  the  GRLL  supports  ten  participant  stations,  but  it  has 
been  expanded  at  times  to  support  fifteen.  The  participant  stations  are  arranged  in  a 
"u-shape”  with  the  open  end  of  the  "u"  facing  the  facilitator  and  front  screen.  The  LAN 
connects  the  participant  stations,  server,  facilitator's  station,  and  a  large  public  screen. 

GRLL  Research  Program 

Research  on  GSS  at  AL/HRG  will  benefit  Air  Force  acquisition  teams  by 
providing  better  tools  and  methods  for  accomplishing  group  work  in  both  face-to-face 
sessions  and  sessions  that  may  be  distributed  in  time  and  place.  The  research  also 
will  explore  the  factors  critical  to  successful  implementation  and  adoption  of  GSS  within 
an  AF  setting.  The  result  of  these  research  thrusts  will  be  the  production  of 
recommendations  for  appropriate  tasks,  tools,  and  procedures  for  more  effective  and 
efficient  group  work. 

An  important  new  group  process  within  the  Air  Force,  and  throughout  the 
Department  of  Defense  (DoD),  is  business  process  reengineering  (BPR).  It  has  been 
codified  within  DoD  by  Directive  81 20.1 ,  which  mandates  its  use  for  development  of 
new  systems.  Within  the  Air  Force,  much  of  the  emphasis  has  focused  on  advanced 
physical  automation  of  information,  commonly  referred  to  as  "paperless  acquisition." 
However,  other  factors,  such  as  the  organization  and  coordination  of  group  work,  may 
also  play  an  important  role  in  improving  organizational  processes.  A  major  focus  of 
research  within  this  area  is  the  computerized  support  of  group  work,  using  GSS. 

Although  commercial  GSS  have  been  available  for  several  years,  they  are  not 
yet  well  integrated  into  mainstream  use  by  most  organizations.  Much  of  what  is  known 
comes  from  a  relatively  limited  number  of  operational  settings,  and  from  experimental 
sessions  undertaken  in  laboratories.  While  there  are  a  number  of  facilities  in  private 
organizations,  there  are  relatively  few  within  the  military  establishment.  The  Navy  has 
an  operational  GSS  facility  at  the  Navy  Yard  in  Washington,  D.C.,  and  the  Army  has 
one  at  Redstone  Arsenal  in  Huntsville,  Alabama,  but  the  Air  Force  does  not  yet  have  an 
operational  facility.  In  addition,  a  review  of  the  literature  clearly  shows  that  it  is  difficult 
to  draw  meaningful  lessons  about  operational  use  from  experimental  settings  [Dennis 
and  Gallupe  1993].  Thus,  relatively  little  is  known  about  long-term  GSS  effectiveness, 
efficiency,  and  user  acceptance  within  operational  settings.  Equally  little  is  known 
about  optimal  designs  and  procedures  for  using  such  systems.  Therefore,  there  is  a 
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need  for  more  investigation  of  operational  GSS  settings  and  sessions,  particularly 
within  the  Air  Force  environment. 

There  are  a  number  of  areas  where  GSS  applications  may  be  particularly 
appropriate  for  the  Air  Force.  Among  these  are  quality  improvement  programs  such  as 
Integrated  Weapon  System  Management  (IWSM),  Quality  Air  Force  (QAF),  and 
Integration  DEFinition  (IDEF)  modeling  of  Air  Force  systems,  as  well  as  planning  and 
ad  hoc  problem  solving  in  general.  The  common  denominator  of  these  areas  is  the 
need  for  appropriately  structured,  frank  discussions  among  groups  working  together 
toward  a  common  goal. 

Some  of  the  factors  that  may  be  important  for  these  groups  include  multi- 
disciplined  composition,  differing  rank,  and  familiarity  of  participants  with  both  the 
nature  of  group  work  and  the  system  used  to  assist  them.  In  each  of  the  areas  that  may 
be  well  served  by  GSS,  there  is  a  need  to  investigate  the  use  of  current  GSS,  create 
and  test  new  tools,  and  design  and  test  new  procedures.  As  GSS  technology  matures, 
the  factors  that  promote  GSS  dissemination  and  acceptance  within  the  Air  Force 
community  will  also  become  important. 

The  ALyHRG  Division  research  program  intends  to  foster  research  into  these 
themes  as  a  natural  component  of  re-engineering  the  acquisition  process.  Research 
into  coordination  and  collaboration  of  Integrated  Product  Development  (IPD)  teams  will 
successfully  augment  current  research  Initiatives  in  other  parts  of  the  AF  and  DoD.  We 
believe  the  GRLL  research  program  will  provide  valuable  information  that  can  be  used 
to  enhance  work  within  the  IPD  community  and  throughout  the  Air  Force. 

Initial  GRLL  Use 

The  GRLL  became  operational  during  February  1993.  Since  we  intended  to 
explore  the  effects  of  GSS  on  real  world  groups,  initial  efforts  focused  on  introducing 
the  GRLL  to  various  collaborative  groups  at  WPAFB.  Initial  publicity  of  the  availability 
of  this  service  was  aided  by  an  article  explaining  the  GRLL  in  the  Skywrighter,  the 
WPAFB  newspaper.  The  article  generated  many  inquiries  and  a  number  of 
demonstrations  and  sessions  resulted  from  it. 

While  Initial  use  of  the  GRLL  for  real  world  sessions  was  slow  at  first,  it  grew 
steadily  during  its  first  year.  We  defined  a  session  as  a  single  use  of  the  GRLL  if  the 
session  duration  was  one  day  or  less.  If  the  session  spanned  multiple  days,  each  day 
of  use  was  counted  as  a  session.  One-hundred-seventeen  sessions  were  conducted 
during  the  first  ten  months  of  use  (through  November  1993).  During  the  first  five 
months,  59  percent  of  all  GRLL  sessions  were  demonstrations  to  potential  users  or 
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potential  champions,  those  who  could  influence  others  to  use  the  facility,  while  the 
remaining  41  percent  were  actual  (as  opposed  to  demonstrative  or  experimental) 
sessions.  During  the  second  five  months,  29  percent  of  nonexperimental  sessions 
were  demonstrations,  while  71  percent  were  meetings.  In  addition,  there  were  2.4 
times  as  many  sessions  during  the  second  five  month  period  (see  Figure  2). 

Experimental  sessions  comprised  those  sessions  held  for  the  purpose  of  testing 
specific  research  hypotheses,  in  which  the  groups  undertook  various  experimental 
tasks,  unrelated  to  real  world  sessions.  Results  of  these  experiments  are  reported  in 
other  papers  on  the  research  program. 


Number  of  sessions  in  the  GRLL  -  First  1 0  Months  (Feb  -  Nov  1 993) 

1st  5  months 

2nd  5  months 

Percent  of  Total 

Demo 

20 

12 

27.3% 

Experiment 

0 

14 

1 2.0% 

0 

8 

6.8% 

14 

49 

53.8% 

Total 

34 

83 

117=  100% 

Figure  2.  Summary  of  GRLL  use 


The  use  pattern  of  the  GRLL  over  the  ten  months  affirms  that  there  is  a 
substantial  market  for  GRLL  services  and  GRLL  research  within  the  AF  community. 

This  is  further  supported  by  the  fact  that  one  of  the  groups  traveled  from  Scott  Air  Force 
Base,  Illinois,  a  distance  of  about  200  miles,  to  use  it.  Many  other  sessions  included 
members  that  traveled  from  other  bases  to  take  part  in  the  meetings. 

Data  were  collected  from  participants  using  a  post-session  questionnaire  (see 
Figure  3).  These  data  suggest  that  GSS  support  is  perceived  by  the  participants  as 
valuable,  with  61 .3  percent  of  the  users  saying  that  the  system  helped  their  process. 
Similarly,  75.5  percent  said  that  they  would  choose  to  use  the  system  again.  These 
findings  are  in  general  agreement  with  findings  from  other  GSS  sessions  held  with  real 
world  working  groups  [Heminger  1989,  Dennis  et  al.  1993]. 

Seventy-one  percent  felt  that  a  distributed  GSS  would  add  even  more  value.  For 
many  of  these  people,  temporary  duty  (TDY)  excursions  to  meet  with  various  working 
groups  are  a  regular  occurrence.  The  cost  in  time  and  money  expended  to  attend 
various  meetings  can  be  considerable.  If  some  of  the  sessions  could  be  held  via  a 
distributed  GSS,  it  would  provide  considerable  savings. 
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An  important  issue  in  the  use  of  GSS  is  the  role  of  facilitation.  Although 
researchers  originally  hoped  to  be  able  to  replace  human  facilitators  with  a  GSS  that 
would  supply  this  role,  our  experience  and  observation  has  shown  that,  in  fact, 
facilitators  are  still  very  important  to  the  GSS  process.  The  data  collected  here  also 
supports  that  conclusion,  with  80  percent  of  the  participants  reporting  that  the 
facilitation  was  very  helpful.  If  these  observations  hold  up  during  future  research,  it 
may  have  ramifications  for  the  design  and  use  of  distributed  GSS,  where  a  human 
facilitator  is  not  easily  incorporated  into  the  system. 


Post  Session  Participant  Assessment  of  G 

RLL  Sessions  (in  %) 

n=  168 

strongly 

disaaree 

disagree 

not  sure 

agree 

1 

GSS  system  helped  our  process 

n 

11.7 

19.7 

38.7 

22.6 

2 

GSS  system  helped  us  reach  our  goals 

0.6 

1.8 

17.3 

67.3 

13.1 

3 

Would  be  more  valuable  if  distributed 

0.6 

3.6 

24.4 

39.9 

31.5 

4 

Facilitation  was  very  helpful 

1.2 

ra 

11.3 

46.4 

33.3 

5 

GSS  has  been  an  obstacle  to  our  process 

34.1 

49.7 

11.9 

3.6 

0.6 

6 

1  would  choose  to  use  the  system  again 

1.7 

4.8 

17.9 

44.6 

30.9 

Figure  3.  Post  session  questionnaire  data 


In  June  1 993,  we  undertook  one  of  the  largest  uses  of  the  GRLL  to  date,  a 
cross-service  depot  maintenance  modeling  effort,  sponsored  by  the  Joint  Logistics 
System  Center  (JLSC).  The  modeling  methodology  selected  for  this  task  was  IDEFO. 
IDEFO  modeling  is  a  process  in  which  the  activities  of  an  organization  are  modeled  in 
terms  of  their  inputs,  their  processes,  and  their  outputs,  as  well  as  capturing  controls 
and  mechanisms.  In  the  past,  this  has  been  a  process  involving  either  individuals  or 
very  small  teams.  The  JLSC  session  was  an  early  attempt  to  support  larger  teams 
doing  IDEF  modeling. 

This  process,  the  largest  to  be  supported  in  the  GRLL  to  date,  involved  30 
participants,  representing  five  service  components  (Air  Force,  Army,  Marines,  Naval  Air 
and  Naval  Sea),  four  IDEF  facilitators  from  MICAH  Systems,  Inc.,  (a  management 
consulting  group),  and  the  services  of  Capt.  Kennon  Moen  and  Dr.  Alan  Hemingerfrom 
Armstrong  Laboratory. 

Although  the  GRLL  was  only  configured  with  15  workstations,  we  accommodated 
30  participants  by  sharing  the  use  of  the  available  keyboards.  This  sharing  was  not  a 
major  obstacle  to  successful  use  of  the  GRLL  for  this  group.  Initially,  JLSC  arranged  to 
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use  the  GRLL  for  one  week  out  of  a  multi-week  process.  After  the  results  of  the  first 
week,  they  were  sufficiently  satisfied  to  ask  to  make  use  of  the  GRLL  on  through  the 
rest  of  the  summer  and  fall  for  their  modeling  effort.  To  date,  JLSC  has  used  the  GRLL 
for  this  process  for  thirty  sessions. 

It  may  be  useful  to  comment  on  the  JLSC  modeling  exercise  because  of  the 
extreme  complexity  of  the  task.  The  task  itself  required  asking  30  people  with  widely 
differing  backgrounds  and  perspectives  to  create  a  common  model  of  the  maintenance 
process  for  all  of  the  maintenance  depots  throughout  DoD.  A  maintenance  depot  is  a 
large  factory  that  handles  maintenance  and  upgrades  for  a  diverse  range  of  products, 
differing  drastically  by  service,  from  tanks  to  airplanes  to  aircraft  carriers.  In  addition, 
each  service  has  its  own  way  of  doing  business  in  areas  such  as  managing, 
scheduling,  and  accounting.  In  essence,  this  project  was  large  and  complex  and 
involved  many  people  with  varied  backgrounds  and  interests. 

In  contrast,  traditional  IDEF  modeling  is  a  process  which  has  been  done  in  very 
small  groups,  with  one  or  two  modelers  talking  to  two  or  three  subject  matter  experts 
(SMEs)  at  a  time.  Individual  interviews  are  then  integrated  by  the  modelers  and 
returned  to  the  SMEs  for  comments  and  corrections,  a  time  consuming  process  under 
the  best  of  conditions.  This  process  is  clearly  not  feasible  for  such  a  large  group  and 
such  a  complex  task.  Therefore,  JLSC  decided  to  make  use  of  the  GRLL  in  an  attempt 
to  include  more  input  from  more  people  in  a  shorter  period  of  time.  Using  the  GRLL  for 
this  process  demonstrated  that  it  can  support  groups  and  tasks  of  this  size.  However, 
because  IDEFO  modeling  is  a  specific  methodology,  and  because  the  GroupSystems 
software  supports  more  generic  processes,  only  part  of  the  modeling  process  could  be 
supported  with  the  current  system.  Within  those  constraints,  however,  the  GRLL  was 
deemed  by  JLSC,  as  well  as  by  the  Armstrong  Lab  personnel,  to  be  an  effective  way  to 
assist  large  groups  in  undertaking  more  rapid  development  of  large,  complex  IDEFO 
models.  An  assessment  by  JLSC  of  the  use  of  the  GRLL  estimated  that  the  work  done 
in  five  months  time  using  the  GRLL  would  have  taken  eighteen  months  or  more  the 
standard  way. 

Other  users  of  the  GRLL  have  included  groups  from  Air  Force  Material 
Command,  Air  Force  Security  Assistance  Center,  Air  Mobility  Command,  Aeronautical 
Systems  Center,  and  Air  Force  Institute  of  Technology.  This  broad  range  of  users  has 
provided  an  opportunity  to  assess  the  usability  and  acceptance  of  this  type  of  group 
support  across  a  wide  spectrum  of  operational  users.  User  assessment  questionnaires 
given  to  each  user  at  the  completion  of  each  group's  work  indicated  a  strong  user 
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acceptance  of  this  type  of  support  for  the  many  tasks  that  brought  the  groups  to  the 
GRLL 

What  Was  Learned 

The  initial  use  of  the  GRLL  has  demonstrated  a  number  of  lessons  that  can  be 
applied  to  both  the  continued  research  effort  in  the  GRLL  and  to  the  operational  use  of 
a  GSS  facility.  The  major  findings  are  described  in  the  following  sections. 

Broad  Interest  In  GSS  support.  Our  experience  with  requests  for  use  of  the  GRLL 
suggests  that  there  appears  to  be  a  strong  interest  in  finding  ways  to  improve  team 
meetings  in  the  Air  Force.  The  only  advertising  done  to  promote  the  use  of  the  GRLL 
was  an  article  that  appeared  in  the  June  18, 1993,  issue  of  Skywrighter.  From  that 
article  and  from  word  of  mouth,  the  GRLL  has  been  kept  very  busy  throughout  the 
calendar  year,  up  to  the  time  of  this  publication. 

Ease  of  learnIng/using.  Across  the  GSS  sessions  held  in  the  GRLL,  there  were  no 
instances  of  participants  unable  or  unwilling  to  use  the  software.  This  may  be  related 
to  the  fact  that  89  percent  of  users  said  they  were  comfortable  using  computers.  Based 
on  observation  and  experience  in  the  GRLL,  the  best  way  to  teach  people  to  use  the 
various  GSS  software  modules  seems  to  be  to  explain  a  module  at  the  time  that  the 
group  first  uses  it.  This  cuts  down  on  trying  to  teach  a  larger  number  of  modules  at 
once,  thus  reducing  cognitive  burden.  For  most  modules,  participants  become 
proficient  with  the  tool  in  a  matter  of  a  few  minutes.  The  participants'  high  level  of 
computer  comfort  probably  helps  to  ease  the  learning  of  the  system.  However, 
everyone  who  tried  to  use  the  system  was  successful,  irrespective  of  his/her  initial 
comfort  level. 

Portability,  The  computers  purchased  for  use  in  the  GRLL  were  specifically  chosen  to 
be  easily  portable  (they  are  all  either  laptop  or  notebook  size).  However,  we 
discovered  there  is  more  to  creating  a  portable  system  than  movable  computers.  The 
state  of  the  art  in  computers  and  networking  still  requires  extensive  cabling  to  power 
and  connect  the  computers.  Then,  once  the  hardware  has  been  connected,  the 
software  must  be  loaded  and  checked  for  proper  configuration.  Therefore,  moving  the 
system  and  setting  it  up  in  another  location  can  be  a  time-consuming  and  laborious  job. 
With  improvements  in  wireless  networks  and  attention  paid  to  custom  cabling,  portable 
systems  may  still  be  an  option  for  some  organizations.  However,  based  on  our 
experience,  we  do  not  recommend  portable  systems  at  this  time. 
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Variety  of  possible  uses.  Although  GSS  have  been  used  successfully  for  many 
different  types  of  groups,  the  common  thread  among  all  of  them  has  been  that  they  are 
working  in  a  text-based  fashion  on  goal-oriented  problems  that  use  the  basic  processes 
of  idea  generation,  idea  organization,  and  selection.  However,  many  other  types  of 
groups  do  not  fit  this  model,  either  because  their  task  is  not  text-based  or  because  they 
need  to  undertake  processes  that  are  not  supported  by  the  system.  For  example,  while 
experience  here  has  shown  that  current  GSS  can  be  profitably  used  with  IDEFO 
modeling  groups,  it  does  so  in  a  limited  fashion.  While  much  of  the  knowledge  needed 
to  create  an  IDEFO  model  can  be  captured  in  a  text-based  GSS  system,  the  specific 
relationships  inherent  in  IDEFO  models  cannot  be  easily  captured  in  a  comprehensive 
way.  In  addition,  IDEFO  models  are  intended  to  be  displayed  graphically,  something 
text-based  GSS  cannot  do. 

Importance  of  pre-meeting  planning.  Based  on  the  previous  experience  of  our 
facilitators,  we  adopted  a  policy  that  all  sessions  would  be  planned  through  a  formal 
procedure  termed  a  preplanning  meeting,  or  "pre-meeting."  At  this  pre-meeting,  the 
facilitators  would  meet  with  the  meeting  leader  to  determine  the  session  goals  and  to 
develop  an  agenda  to  accomplish  that  end.  This  process  worked  quite  well  as  a  way  to 
establish  an  agenda  that  could  be  successfully  implemented  within  the  constraints  of 
the  GSS  mediated  session.  While  we  did  not  run  comparisons  of  sessions  without 
preplanning,  we  believe  that  preplanning  is  an  important  condition  for  a  successful 
GSS  session.  We  initially  expected  that  pre-meetings  would  be  one-time  events  lasting 
about  an  hour  for  each  session,  but  it  turned  out  that  quite  a  few  of  our  sessions 
required  several  pre-meetings.  Answering  such  basic  questions  as  the  goal  of  the 
session,  at  times,  took  multiple  pre-meetings.  Our  experience  with  this  process  has 
convinced  us  that  preplanning  is  essential  for  most  of  the  sessions  that  we  have 
facilitated. 

Because  of  this  observation,  we  have  tentatively  concluded  that  many  people  do 
not  routinely  think  through  their  meeting  processes  in  terms  of  goals  and  processes  to 
achieve  those  goals.  To  the  extent  that  this  is  true,  preplanning  may  provide  value  to 
many  meetings,  whether  supported  by  a  GSS  or  not.  It  may  be  that  this  need  for  better 
meeting  planning  has  been  present  all  along  but  that  it  has  only  become  visible  in  the 
light  of  planning  for  GSS  use. 
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Importance  of  facilitation.  A  distinction  can  be  made  between  the  structure  provided 
by  the  GSS  and  the  facilitation  of  the  process  provided  by  a  facilitator.  The  GSS 
provides  a  somewhat  static,  though  tailorable  structure  for  group  work.  This  structure 
is  based  on  a  series  of  logical  steps  that  are  intended  to  lead  the  group  to  achieve  its 
goals.  Facilitation,  on  the  other  hand,  relies  not  so  much  on  a  logical  structure  of  the 
process  as  it  does  on  the  ebb  and  flow  of  group  interactions  throughout  the  meeting. 
These  two  structuring  processes,  the  GSS  tools  and  facilitation,  work  together  to  aid 
the  group  in  attaining  its  goals.  In  many  of  our  GSS  sessions,  the  facilitators  adjusted 
the  agenda  and  modified  the  use  of  the  GSS  to  respond  to  opportunities  or  to  adjust  to 
problems  that  arose.  Without  proper  facilitation,  the  group  might  not  achieve  its  goals 
and  would  likely  be  unable  to  capitalize  on  opportunities  that  may  arise.  Our 
observation,  as  well  as  comments  from  the  participants,  leads  us  to  believe  that 
facilitation  is  an  important  factor  to  the  successful  use  of  GSS. 

Ability  to  support  complex  projects  with  biend  of  manual  and  automatic 
techniques.  Although  the  GSS  software  that  was  in  use  in  the  GRLL  was  not  designed 
specifically  to  support  IDEF  modeling,  it  was  used  effectively  as  a  part  of  an  overall 
process,  which  included  both  manual  methods  and  a  stand-alone  computer  modeling 
system.  We  learned  that  GSS  do  not  have  to  be  the  only,  or  even  the  primary,  vehicles 
for  group  interaction  for  them  to  provide  value  to  a  group  endeavor.  The  key  to  making 
effective  use  of  GSS  appears  to  be  identifying  those  processes  within  the  larger 
session  that  can  be  supported  and  augmented  by  GSS. 

Need  for  oral  as  well  as  computer  Interaction.  We  have  observed  that  conversation, 
as  well  as  computer  input,  is  important  to  participants  of  a  GSS  session.  While 
structured  information  is  entered  into  the  computers  for  capture,  much  oral  discussion 
still  takes  place  in  a  GSS  session.  It  will  be  important  for  designers  of  GSS  facilities  to 
keep  this  need  in  mind  as  they  design  future  GSS  environments.  This  will  also  have 
ramifications  for  designers  of  distributed  GSS,  where  the  participants  will  not  always  be 
meeting  face-to-face.  Additional  communications  channels  may  need  to  be  developed 
to  support  some  type  of  informal  "oral"  interactions. 

Need  for  extensions  of  the  system.  While  the  basic  processes  of  gathering 
information,  organizing  that  information,  then  making  choices  about  it  are  well 
supported  by  current  GSS  software,  there  are  specialized  needs  of  some  groups  that 
could  be  better  supported  by  software  that  is  tailored  to  those  needs.  An  example  of 
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this  is  the  use  of  GSS  for  some  types  of  IDEF  modeling.  There  are  specialized  tasks 
that  take  place  in  an  IDEF  session  that  are  carried  out  by  the  team  but  are  not 
supported  by  the  current  text-based  GSS.  These  tasks  include  the  ability  to  create 
complex  process  structures  and  the  ability  to  display  them  graphically. 

Potential  Future  Research  Directions 

IDEF  modeling.  DoD  Directive  8020.1  mandates  that  functional  modeling  will  be  used 
to  create  as-is  and  to-be  models  of  systems  that  are  to  be  supported  by  new 
information  systems.  More  recently,  DoD  has  stipulated  that  the  method  for  doing  that 
will  be  IDEF  modeling.  However,  the  current  process  for  implementing  IDEF  modeling 
tends  to  be  iengthy  and  labor-intensive.  Work  in  the  GRLL  has  shown  that  the  IDEF 
modeling  process  can  be  made  more  effective  and  efficient  with  GSS  support,  but  much 
can  be  done  to  tailor  this  technology  for  IDEF  modeling.  Research  in  this  area  could 
provide  substantial  cost  savings  and  increased  value  to  the  modeling  process  but  will 
require  the  development  of  both  new  techniques  for  IDEF  modeling,  as  well  as  the 
design,  development,  and  testing  of  new  GSS  tools  that  can  augment  the  IDEF 
modeling  process. 

Distributed  group  work.  Based  on  feedback  from  many  GRLL  users  ,  we  believe 
there  is  a  need  for,  and  an  interest  in  using,  a  distributed  GSS.  Although  a  few  of  the 
major  GSS  systems  are  beginning  to  explore  this  need,  much  is  still  to  be  learned. 

There  is  a  real  need  for  continued  research  to  investigate  the  best  ways  to  support 
teams  that  are  working  as  a  group,  even  though  some  of  their  "meetings"  may  involve 
virtual  connections  in  which  the  members  may  be  participating  from  different  locations 
at  different  times.  Downsizing  and  IWSM  operations  make  distributed  meetings  more 
desirable  than  ever  before.  Advances  in  GSS  supporting  technologies  (e.g., 
networking  and  communications,  multi-media  presentation)  further  extend  the  feasibility 
of  creating  effective  distributed  GSS. 

Automation  of  facilitation.  The  movement  toward  distributed  GSS  will  necessitate 
that  new  ways  be  developed  to  provide  needed  meeting  facilitation.  By  its  very  nature, 
a  distributed  session  has  participants  meeting  at  different  times  and/or  places,  using 
the  computer  network  to  provide  the  organization  and  structure  of  the  "meeting.” 
Therefore,  it  will  not  be  possible  to  have  a  facilitator  stand  up  in  front  of  the  group  to 
help  with  group  interactions  and  to  help  move  the  agenda  along.  Not  only  may  people 
not  be  meeting  at  the  same  place,  but  they  also  may  not  even  be  meeting  at  the  same 
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time.  This  will  necessitate  that  the  meeting  will  last  longer  than  a  comparable  same 
time  and  place  meeting.  It  is  conceivable  that  such  a  virtual  meeting  may  take  place 
across  several  days,  so  providing  "live"  facilitation  becomes  an  issue.  Without  a  live 
facilitator,  any  facilitative  process  may  have  to  come  from  the  system  itself.  We  do  not 
yet  know  the  important  features  of  this  process,  nor  the  method  of  implementing  them  in 
a  distributed  GSS. 

Inclusion  of  expert  system  support.  Although  each  meeting  is  unique,  similarities 
among  meetings,  such  as  stringing  together  sub-processes  to  reach  the  group  goal  and 
the  sequencing  of  those  steps  to  get  the  best  results,  exist  as  well.  Which  steps  to 
take,  in  what  order,  and  how  to  map  those  steps  to  the  various  automated  and  manual 
support  tools  is  largely  a  matter  of  heuristic  knowledge.  A  group  may  be  able  to  reach 
its  goals  in  many  different  ways,  but  some  will  likely  be  more  effective  or  efficient  than 
others.  Thus,  when  planning  a  meeting,  an  expert’s  rules  of  thumb  can  be  very  helpful. 
This  type  of  knowledge  can  be  captured  in  an  expert  system  and  can  be  made  widely 
available  to  support  GSS  processes. 


Summary 

The  GRLL  has  been  successfully  designed,  built,  and  implemented  for  the  study 
of  the  use  of  computerized  support  for  problem-solving  groups.  Initial  use  has 
supported  our  belief  that  many  types  of  AF  groups  could  benefit  from  the  support 
provided  by  GSS  technology.  Increasing  use  of  the  GRLL  by  repeat,  as  well  as  new, 
customers,  suggests  that  there  is  a  real  need  for  the  type  of  support  provided  by  this 
facility.  The  use  of  the  facility  to  date  has  provided  much  insight  into  the  specifics  of 
AF  use  of  GSS  technology  and  has  revealed  a  number  of  potentially  fruitful  areas  of 
interest  for  future  research.  Two  of  the  most  promising  areas  are  the  extension  of  GSS 
support  to  distributed  meetings  and  the  extension  of  GSS  support  to  other  types  of 
structured  meeting  processes  such  as  IDEF  modeling. 
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